1. Field of the Invention
The present invention relates to the field of quality evaluation of audio signals and particularly to quality evaluation systems suitable for practical use for everyone.
2. Description of the Related Art
Today, standardized perception-based measurement methods (perceptual measurement) are used for measurement-based evaluation of the quality of coded voice and audio signals. Known methods are the so-called PESQ method (PESQ=perceptual evaluation of speech quality) described in the standardization document ITU-T P. 862 (February 2001). Another known measurement method for quality evaluation is the so-called PEAQ method (PEAQ=objective measurements of perceived audio quality), which is presented in the standardization document Rec. ITU-R BS. 1387-1 (1998-2001). What these methods and/or further methods for quality evaluation have in common is that a signal to be tested (“test signal”), which is generally the output signal of a system or network or generally of an elements to be examined (DUT), is compared with an original or also reference signal, which is generally the input signal into the DUT to be tested.
Such a general “setting” is illustrated in FIG. 4. The original audio signal fed into a DUT 600 represents the reference signal or input signal, while the output signal after the DUT 600 is used to either perform a subjective hearing test with test persons, such as indicated by a subject 602, or to perform a quality evaluation method, such as PESQ or PEAQ, such as indicated by a model 604. By supplying the output signal from the DUT 600 to the subject 602, it is thus possible to perform a subjective hearing test typically performed with several test persons in standardized rooms. By supplying the original audio signal before the DUT 600, i.e. the reference signal, and the audio signal distorted by the DUT to the model 604, an objective test, i.e. an algorithmic evaluation without subjective test persons, may be performed.
The DUT 600 is typically a system whose influence on audio quality is to be evaluated. Such a system is, for example, a telecommunication connection and particularly a telephone connection that may be wireless or wired. An alternative DUT 600 is, for example, a coder/decoder path to evaluate the quality interference of a coding concept with downstream decoding concept. If the model operates in the intended way, the output of the model is supposed to be a prediction of the perceived quality that test persons would subjectively mark on a scale when hearing the output signal of the DUT 600.
In the case of the PESQ method, for example, the original audio signal, i.e. the audio signal before the DUT 600, which is the reference signal, is compared with the audio signal distorted by the DUT 600 considering a time delay, wherein a psychoacoustic model is used. In particular, both the original audio signal before the DUT 600 and the distorted audio signal after the DUT 600 are transformed into a so-called internal representation which is analogous to the psychophysical representation of audio signals in the human hearing system, wherein there are particularly considered parameters such as the Bark scale and the loudness (sone), as it is known in the art. The internal psychophysical representation of the original audio signal is then compared with the internal psychophysical representation of the distorted audio signal to calculate one or more error parameters, depending on the model, that allow a quantitative quality statement.
A quality evaluation method illustrated based on FIG. 4 is also referred to as “intrusive” method, because it is necessary to feed the reference signal, i.e. the original audio signal, into the system to be tested (DUT 600). At the output of the DUT, there is then obtained, as described above, the test signal to be evaluated, which is also referred to as distorted audio signal in FIG. 4 and/or generally as audio signal. The output of the DUT 600 may, for example, be the distant end of a telephone connection of two parties, wherein the original audio signal is fed in at the near end as reference signal. In this case, the measurement method, such as PESQ, would characterize the voice quality of a telephone connection.
As described, the algorithmic measurement methods are based on a combination of psychoacoustic and cognitive findings about the human hearing perception. The underlying experiment of these methods first consists in performing a subjective hearing test, in which a statistically sufficient number of test listeners (“subjects”) is presented with a series of voice and/or audio sequences for evaluation. The testers evaluate these sequences by means of a discrete and/or continuous quality scale, which is also referred to as “opinion scale” in the art and ranges, for example, from 1 (“bad”) to 5 (“excellent”). Such subjective hearing tests are, for example, presented in the standardization document ITU-T P.800 (August 1996).
Consistently, test sequences are prescribed in standardized test procedures, such as in the standardization document Rec. ITU-R BS.1116-1 or Rec. ITU-R BS.1534, that have a duration of typically between 8 and 12 seconds, whose maximum length, however, does not exceed 20 seconds. Although these test sequences are real signals, they are not stochastic and/or randomly originating from a real scenario, but standardized predetermined test sequences that may be fed into the DUT to be considered in an experiment to gain the test input signal, i.e. the audio signal distorted by the DUT.
Recently, developments have been presented that allow to perform non-intrusive tests, too, which are supposed to allow an estimation of the voice quality exclusively based on an analysis of the test signal on the reception side, i.e. without feeding in a reference signal on the transmission side. Such developments are particularly advantageous for practical realizations, because they allow, for example, a statement on the voice quality of a mobile radio connection solely in the terminal without requiring any measurement arrangements or measures and/or manipulations in the telephone network for feeding in a reference signal. Each real telephone conversation may thus, in principle, be subjected to a quality evaluation with such a non-intrusive concept.
Since both intrusive measurement methods and particularly non-intrusive measurement methods by now provide significant results, as it is also documented by the quickly advancing standardization, there is still the problem how such a system may be made available to the general public.
Non-intrusive measurement methods lend themselves better to this purpose, because they at least require only the output signal of the DUT channel 600 (FIG. 4), while intrusive measurement methods additionally require the original audio signal not distorted by the channel.
In addition, the audio signal distorted by a channel has to be provided to the measurement device for a quality evaluation of audio signals, wherein at the same time the quality evaluation concept is to be available not only for a few special applications, such as in a laboratory of a telephone company, but that, in principle, everyone may access the inventive quality evaluation concept to be able to perform a quality evaluation of audio signals they generated.